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ABSTRACT 

The Comprehensive School Mathematics Program (CSMP) 
is a program of CEMREL, Inc., one of the national educational 
laboratories and is funded by the National Institute of Education. 
Its major purpose is the development of ~ curriculum materials in 
mathematics for grades K-6. Beginning in September, 1973, CSMP began 
an extended pilot trial of its Elementary Program. This report is an 
attempt to summarize, in a reasonably non- technical wa,y, the 
information collected during the first year of the evaluation. 
Included are: (1) descriptions of trial sites; (2) results of test 
data including standard content, cognitive skills, and CSMP content; 
(3) teacher reactions to CSMP; (4) costs; (5) external review; (6) 
school decisions regarding continued use; and (7) a discussion of the 
pilot trial. (RH) 
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Description of Evaluation Report Series 



The Comprehensive School Mathematics Program (CSMP) is a program 
of CEMREL, Inc, one of the national educational laboratories, and is 
funded by the National Institute of Education. Its major purpose is 
the development of curricula^) materials in mathematics for grades K-6. 

Beginning in September^ 1973, CSMP began an extended pilot trial 
of ics Ei.ementary Program. The pilot trial is longitudinal in nature; 
student? who began using CSMP materials in kindergarten or first grade 
in 1973-74 wil] use them in first and second grade respectively in 
1974-75 and in second and third grade the fo J lowing year. Hence the 
adjective ''extended/* The limited scope of these trials does not 
justify the tern ''field trial" since the major focus of the evaluation 
is on a limited number of classes in the metropolitan St, Louis area. 

The evaluation of the program in this extended pilot trial is 
intended to be reasonably comprehensive and to supply information 
desired by a wide variety of audiences. For that reason the reports 
in this series are reasonably non-technical and do p.ot attempt to 
widely explore some of the related research issues. The list of 
reports fiom 'che first year of the extended pilot trial is given on 
the next page. Th.e most comprehensive of these are 1-A-l: Overview, 
Design and InstrurentaL ion and l-A-5: Pinal Summary Report, Year 1. 
The former will be particularly useful to the reader in providing a 
description of the program, the philosophy and goals of the evaluation 
and the relationship of indivicual reports to the evaluation effort as 
a whole. 
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Key to Indexing 
Observations of CSMP First Grade Classes 

''2'' refers simply to the number within a given year and type of data 

**C'* refers to the type of data being reported 

A: Overview, summary and theoretical reports 
B : Student outcomes 
C: Non-test data 

''V' refers to the year of the Pilot Study according to the following: 

# • » 
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Introduction 

In the fall of 1973, kindergarten and first grade materials of the 
Comprehensive School Mathematics Program began being used in over 100 classes 
as part of CSMP's Extended Pilot Test. This Pilot Test is longitudinal in 
nature; most kindergarten and first grade classes beginning the program in 
1973-74 are continuing with first and second grade materials in 1974-75. 

During the first year of this Pilot Trial a considerable amount of 
information was collected about the program, how it was being used and 
with what results. The list of Evaluation Reports on page iv will give the 
reader some idea of the variety of evaluation tasks c;^rried out. With the 
exception of the first report, 1-A-l, each report deals with a specific issue 
or set of data. Report 1-A-l is intended to be an introduction to the series 
and describes the program in terms of its content and materials and its 
history, the general and specific objectives of this summative evaluation, 
and the setting and design of the Pilot Trial. 

The present report is an attempt to summarize, in a reasonably non- 
technical way, the infomation collected during this past year. While it 
is not possible, nor necessarily desirable, to suppress the opinions and 
prejudices of the evaluator, one hopes the reader can easily separate the 
presentation of data from the author's interpretation of those data. In 
any case, if the reader wishes more information about certain of the data 
reported here or would like to see the actual tests or questionnaires or 
instrujrnents used, he or she may consult the appropriate report in this 
Evaluation Report Series. 
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Setting 

"Local** and "Outer Ring^^ Site s 

As a result of rather moderate publicity of the CSMP Extended Pilot Trial, 
29 school districtc decided to try out the CSMP kindergarten and/or first grade 
materials. No conditions were placed on the number or location of pilot classes 
so that participating school systems were free to use^ the materials in as 
limited or extensive a wa/ as they wanted and with whatever kinds of classes 
or teacher^i they wanted. They were required to pay the cost of producing the 
necessary instructional materials, to provide or allow for the collection of 
relevant evaluation data and to provide a coordinator for the program in their 
district. This coordinator was responsible for overseeing the implementation of 
the program including the training of pilot teachers (except as noted below) and 
was generally the liaison person between CSMP and the local district. The 
coordinator was also required to attend a one-week training workshop in the use 
of the CSMP curriculum. 

Eight of the 29 participating school districts were located in the St. Louis 
area and were designated "local" as opposed to "outer ring." For them some 
additional conditions were imposed. First, for each first grade pilot class a 
suitable comparison class was to be established and accessible for the collection 
of various data. Second, it was understood that a considerable amount of 
"evaluating''' would be done in these local classes, including interviews, 
observations, and group and individual testing. Third, all local pilot teachers 
were trained at one of the series of one-week summer training 'workshops conducted 
by CSMP. Any further periodic training sessions to be held during the school 
year were the responsibility of the local coordinators. 

Thus local classes tried out the materials in fairly well controlled 
circumstances with standardized training, comparison ciasseS)and considerable 
observation, while classes at distant sites in the "outer ring" implemented the 
program virtually without restriction. 

C oord ina tors and Teacher Training 

The people who became CSMP coordinators could be put into one of lour 
categories according to their position within the school district. Five wore 
regular kindergarten or first grade teachers who filled the double role of 
pilot teacher and coordinator. They usually had received between nine and 
15 semester hours of college math credit. Five coordinators were elementary 
school principals whose collective math background was about the same as the 
teacher-coovdinators. Five coordinators were primarily college based personnel 
who worked with a school district on a somewhat ad hoc basis in implementing 
this pilot trial. All five had advanced degrees in some area, though two had 
less than 15 semester hours of college math. Tne largest group of coordinators, 
13, were district supervisory officials, usually with a title such as curriculum 
coordinator or curriculum consultant. This group, like the college-based 
coordinators, tended to have an advanced degree, although again about half (6) 
had less than 15 semester hours of mathematics credit. 
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Thus there was considerable variation in the role the coordinator 
ordinarily played in the school system, in the authority over pilot teachers 
which that role provided and in the mathematics background of coordinators. 
Based on rather informal impressions, it is the author's opinion th^T the 
coordinator's ability to successfully coordinate the imple^^aentatior. of CSMP 
on a pilot basis was not dependent on either his or her role in the district 
nor on mathematics background. Both , effective and relatively ineffective coor- 
dinators were found in each of the four job categories and regardless of 
mathematics background. 

As noted previously, all coordinators, as well as all pilot teachers in 
the St. Louis area, attended a one-week training workshop in the summer. The 
workshop time was ajout equally divided between the development of mathematical 
content and the methods used to teach the content, including the use of 
various instructional materials. The presentation was in lecture format for 
the most part with some question-and-answer sessions. Experience^ teichers 
were available to help answer questions and specially prepared films were also 
used. A more complete description of the workshop is given in another Evaluation 
Report.* 

Upon return to their local district, outer ring coordinators were 
responsible for the training of their own pilot teachers. The extent of this 
second level training varied immensely from district to district. Where the 
coordinator was a principal, teacher or university based person, the program 
was usually being piloted by ? or 3 teachers in one school only and no formal 
training sessions were held prior to the beginning of school. During the 
school year, some coordinators held scheduled sessions at definite times while 
others simply got together with the teachers "when the need arose." In one or 
two cases virtually no training was given at all. 

Where the coordinator was a district-wide supervisor of some kind, such as 
a "curriculum consultant" or where more than 1 school was involved, formal training 
sessions were usually held prior to the beginning of the school year. These 
sessions ranged from 4 to 20 hours in duration with an average of about 7 or 8 
hours. In addition, sessions during the year ten'ed to be more regular than 
for other coordinators, generally about once per month. 

One reaction common to many teachers and coordinators was the difficulty 
in having a successful workshop when many of the instructional materials were 
not available. The fact that materials did not arrive in some schools in proper 
time for student usage, let alone for teacher training, caused some difficulties 
for all involved. To put it mildly, occasional annoyance was expressed. 

The ^'Local^' Setting 

Teachers in the local St. Louis area were trained directly by CSMP 
personnel in the summer workshops described previously. (In fact the workshop 
sessions generally had in attendance a mixture of both local and outer ring 
coordinators and local teachers.) During the course of the school year 
inservice training for these local teachers was the responsibility of the 
coordinators. 



*Evaluation Report 1-C-l: Teacher Training Report 
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Altogether 15 kindergarten and 17 first grade teachers ' attended one of the 
summer workshops. Most had an .undergraduate major in education and six of the 
32 hv^u a Master's Degree. About a third had each of five years or less teaching 
experience, between 6 and lO^ears, and more than 10 years experience; the 
kindergarten teachers as a group aver." ,ing 13 y^xrs of experience and the first 
grade teachers about 7.5 years. Altogether, rive teachers had more than nine 
semester hours of credit in mathematics Ji r \;h education with the kindergarten - 
teachers averaging about oh credits and :rl7\.l grade teachers about 7^ credits. 
Teachers were asked to rank their favor .tc^ subjects to teach. Six of the teachers 
ranked mathematics first, 13 ranked it second, nine ranked it third, and four 
ranked it fourth or lower. Thus, as a ;iroup, local teachers seem to have been 
fairly representative of kindergarten and first grade teachers in general, at 
least based on this information. 

A description of the various local CSMP classes is given in Table 1 below. 



Tablo 1 

Description of Local 
CSMP Pilot Classes 





Kinder f. art en 


First 


Grade 


PredtMinaat 
Racial Makeup 


Estimated Socio- 
Econonic Status* 


Type of 
CoMunity 


Dlatrlct 


c 

i 

H 
0 

u 

1 


• 

w 

M 

m 

u 

u. 
O 

tm 

Xi 

§ 


>«« M 

0 M 
(« 

U ^ 

*> u 

1 2 

ac c 

51 


« 
« 

u 
o 

U 
«) 

i 

a: 


Mean Number of 
StuJents/CU%» 


Black 


White 


Mixed 


Low 


Low-Middle 


Middle 


• 

1 

z 




Perfiuon- F lo r i s s an t 


2 


4 


19 


2 


24 




/ 








/ 






Suburban 


Prsncis Howell 


0 






5 


25 




/ 






/ 


/ 






Rural Suburban 


H«rrin 


1 


2 


23 


1 


26 




/ 






/ 


/ 






Snail Town 


Noxaandy 


2 


4 


28 


4 


22 






/ 




/ 




/ 
f 




Suburban 


Gt. Louis 


4 


7 


23 


4 


32 


/ 




/ 


/ 


/ 










Udue 


3 


S 


17 


n« • 


23 






/ 












Suburban 


SMser 


2 


3 


26 


3«-- 


23 




/ 






/ 


✓ 






Small Town 


Affton 


1 


2 


23 


0 






/ 










✓ 




Suburban 



'Based on subjective impressions of the comaunity in which 
s school with pilot classes was located. 

**Started teaching the progran 2 nonths into the school year, 
■ot trainad in CSMP siMtr workshop. Not used for ca«parativ« studies. 

***Tau(hr. by the saae teacher. Only first gnda classes in the district. 
Not used for coaparative studies. 



Coanparison Classes 

Except in Ladue and Se.sser, a comparison class was established for each local 
first grade CSMP class. The usual situation was that, within a school, one group 
of first graders became a CSMP class and another (or the other) group became the 
comparison class. The assignment of students to CSMP and comparison classes was 
not ;;trictly random but in all cases the school principals attempted to make the 
pairs of classes as equal as possible in numbers and in compsoition of students. 
In four cases the CSMP class was the only first grade class in the school and the 
comparison class vras then drawn from the nearest adjacent school. 
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Near the beginning of the school year the Cognitive Ability Test 
(Houghton-Mifflin, 1968) was administered to all CSMP and comparison first 
grade classes. This is a 75 item test designed to assess the development 
of cognitive abilities. It is the dovmward extension of the Lorge-Thorndike 
Tests and the four specific areas of cognitive skills measured by the test 
are, according to the test manual: 

1. The ability to label or name objects or actions or to identify 
objects when given their use, 

2, The ability to identify size, position, and quantity, 

3, The ability to see relationships and, to categorize or 
classify objects, 

4. The ability to deal with quantitative relationships a:id 
concepts. 

The results of this test were used to adjust end-of-year test scores for 
differences in entering ability, ie. , as a covariate. It is referred to as the 
"Pretest*' in this report but it was not a pretest in the sense of a pre-post 
administration of the same achievement test. The actual adjustments were 
rather small since, as measured by this test at least, the CSMP and comparison 
ciaf^es uere very similar in entering ability. Although there were occasional 
differences within a pair of (CSMP-comparison) classes, the mean raw pretest 
score across the 16 CSMP first grade classes was 49,0 and across the 16 
comparison classes was 49,2. 

Thus,* while assignment of students to CSMP and comparison classes was not 
random, pairs of classes appeared to be fairly comparable in important indices 
such as entering ability, academic background and socio economic status. 
Similarly, although the teachers were not randomly assigned to CSMP, there is ' 
reason to believe that, as ^ group, the comparison teachers were as skillful 
as the CSMP teachers. The principals of 10 schools in which both a CSMP teacher 
and a comparison teacher taught were asked whether or not the comparison teacher 
"would do as good a job teaching CSMP" as did the CSMP teacher. Nine affirmative 
responses were given and the tenth response was that the comparison teacher would 
probably do a slightly better job These responses agree with the subjective 
impressions of the author, accumulated during the course of the year. 

The mathematics curriculuon used by the comparison teacher? was whatever 
the school or school district was then using. Tl-.is turned out to be one of 
four commercial math series: 



Modem School Mathematics; Structure and Use , I , 

Duncan, Ernest R. , et al, Houghton-Mifflin Co., Boston:; 1967, 

Elementary School Mathematics Book 1 , Second Edition, 
Eicholz, Robert E. , Addison Wesley Publishing Co,, 
Menlo Park, Calif, 1968, 

Silver Burdett Mathematics 1 , LeBlanc, John F., et al. 
General Learning Corp. , Morristown, N.J . , 1973. 

Essential Modem Mathematics, Primary B, Glennon, Vincent J,', 
et al, Ginn and Co,, Lexington, Mass,, 1972, 
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These texts were very similar in overall content coverage (See p. 
this report). 
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* 'Out ex- Ring" S ites 

A total of 79 teachers taught CSMP math to 98 classes in outer ring sites, 
A list of the numbers of participating teachers and classes for each of 21 sites 
is given below. There is really no way to adequately summarize the diverse ways 
in which the program was implemented in these outer ring sites. 



'*Loc»l" School Districts 



NiMber of Kindergarten 
Classes 



Number of First 
Grade Classes 



Total Number of 
Pilot Teachers 



St. Louis, Mo. 




4 8 


Frances Howell, Mo. 




5 5 


Ferguson-Florissant, Mo. 




2 4 


N'onsand/, Mo. 




4 6 


Affton, Mo. 




0 1 


Sesser, 111. 


3 


3 3 


Herrin. 111. 


2 


1 2 


Ladue, Mo. 


5 


2 5 



Nufliucr of )CinJcr£aitcn 


Number of First 


Total Number of 


"Outer king" Scliool Districts Classes 


Grade Classes 


Pilot Teachers 


Andrews, Texas 2 




4 


Austin, Texas * 4 




6 


B«thl«hcin, Penn. 7 




7 


Cmlifornia State College, Penn. 1 




2 


C«dartown, Ga. 5 




7 


Central Washington State Collece. Wash. 2 




3 


Clark$vilie-Montgoncr>' Co., Tenn. 1 




2 


Detroit, Michigan ) g 




14 
2 


Elixabethtown, Ky. / 2 




Fort Campbell Dependent Schools, Ky. ( 2 




^ 


Lincoln County, Tenn. j 2 




4 


Kmrquette Diocesan Office of Educ, Michi 1 




2 


Middleton Towjship, N.J. ? 4 




2 


Nashville, Tenn. 1 




2 


New Hartford, N.Y. 2 




2 


North Allegheny, Penn. 5 




6 


Philadelphia, Penn. 0 




4 


Portland, Maine 4 




3 


Shipponsburg State College. P^nn. 




2 


Tennessee Tech. University, Tenn. 1 






Mhisnan, Cal. } 


0 


3 
1 
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Results of Test Data 



Tests of Standard Content: First Grade* 

An analysis was made of five of the largest selling commercial math 
series** in order to determine what skills and concepts are generally taught 
to first grade students. Emphasis was placed on the actual tasks provided 
for students, usually in the form of practice exercises. For each content area 
a sequential list of tasks as they appeared in the text was compiled as well 
as the exercise format, sample exercises (to illustrate difficulty levels), 
and nxomber of pages. The relative emphasis of various content areas is 
illustrated in Table 2, which was derived from the five content analyses. 

Table 2 

Percentage of Pages Devoted to 
Each Content Area for Five 
Commercial First Grade Mathematics Texts 









Texts 






Content Areas 
















, ] 


2 


3 


4 


5 


Mean 


Addition and Subtraction 
Counting, Nxomeral Recognition 
Order (<,>), Ordering Numerals 
Place Value 

Matching Sets of Objects 
Measurement (linear, time, money) 
Geometry (curves, fractional parts) 
Miscellaneous 


27 
24 
9 

6 

5 
28* 


40 
25 
8 
8 
3 
8 
8 


48 
9 

il 
8 
6 
9 
1 

10 


38 
12 
14 
4 

8 

7 ■ 
18** 


42 

27 
9 
5 
3 
7 
6 
1 


39.0 
19.4 
10.2 
5.0 
3.6 
6.4 
5.4 
11.4 



*ColoT, pattern recognition, classification; ie. readiness s 



♦♦Number line, counting by x, number properties. 



♦Evaluation Report l-B-3: End-of-Year Test Data: Standard 
First Grade Content 



**The four titles on page 5 and One by One; Harbrace Mathematics 
(Blue), Payne, Joseph N., et al , Harcourt Brace Jovanovich, New York, 1972 
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The siailarity between texts is even greater than shown by Table 2 since 
within content areas, the specific objectives, the item formats and the difficulty 
levels were very similar. Text 1 in Table 2 is different from the others in first 
allocating time to certain readiness skills before preceding to the usual skills 
and concepts and can be considered as not getting as far along in the standard 
first grade sequence. 

Based on this analysis « 8 test scales were constructed, each one covering 
a different content area ar.d generally using the kind of itaa format commonly 
found in the text books. For each local first grade CSMP and comparison class, 
a random half of the students took part of the test; the other half of each 
class took the rest of the test. Testing time was about 35-40 minutes for 
each half and students were tested in groups of 10 to 12 at a tims. 

For each test scale, the class mean was calculated for each of the 16 
comparison classes. Table 3 shows the resulting data using an analysis of 
covariance procedure which adjusted scores for differences in entering ability 
as measured by the pretest. The scale labelled "Applications" is for simple 
word problems and the scale labelled "Measurement" is for a conglomerate of 
items involving telling time, coin recognition, identifying shapes, fractional 
parts and physical measurement. The items in the scale "Larger Numbers" were 
scattered throughout the two halves of the test and involved addition, subtraction, 
place value - familiar areas - but using numbers larger than in the other 
scales. 

Table 3 

Analyses of Class Means 
for Tests of Standard Content 







Number 


Adjusted Mean Scores* 


F TestA* 


Subtest 




of Items 


CSM" 


Non-CSMP 


P less than 


Numerat i on 




12 


9.8 


9.5 


.22 


Subtraction 




12 






.13 


Order 




8 


5.6 


5.^ 


.52 


Appl 1 cat ions 




7 


5.0 


5.0 


.97 


Larger Numbers 


(A) 


7 


3.0 


2.3 


.01 


Larqer Numbers 


(B) 


k 


0.9 


O.k 


.01 


Place Value 




7 


3.^ 


2.8 


.06 


Measurement 




]k 


8.5 


7.8 


.07 


Addi t ion 




13 


10.2 


9.9 


.53 



* Adjusted for entering ^ability based on pretest 
** 1 and I'* degrees of freedom 
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It can be seen that CSMP classes had significantly (p<,05) higher scores 
on the Larger Number scales and the difference approached significance on two 
other scales, Place Value and Measurement. The only scale on which comparison 
classes had higher scores was the Subtraction scale and the difference was not 
significant. 

As part of a cooperative venture these test scales were also administered 
to the seven CSMP classes and seven comparison classes in Region V of the 
Detroit School System,* The analysis described above was repeated with 
slightly different results. All differences were ir favor of CSMP but the 
difference was significant on the Applications scale an<i approached 
significance (p<,07) on the scales Measurement and Addition, 

Test of Cognitive Skills* 

Three tests were constructed. They were content-free in the sense; that 
neither CSMP nor comparison classes had been exposed to similar kinds of tasks. 
However, the ideas and thinking required to perform them were thought to be 
related to some of the concepts stressed in the CSMP program. Each test was 
individually administered by specially trained testers, requiring about 10 to 
IS minutes per student. Each test was administered to each student in from 
4 to 6 CSMP classes and to their paired comparison classes, with no class 
being given more than one test. 

Test 1, Classification Test". A s^t of nine identical looking black 
rubber balls was used. Some were identical and some 
differed along dimensions of weight and/or the sound: they 
made when picked up and shaken. Students were asked 
questions about these attributes including some dealing with 
set-subset concepts, without of course using those words, 

CSMP students had some limited experiences with attribute blocks, with 
classifying and with working (intuitively) with sets and subsets. Thus this 
test was designed to measure whether or not these experiences better equipped 
them for classification tasks like those on this test. 

Test 2. Relations Test: There were three subtests dealing with Relations, 

a) Faster Than Test: Students were shown three pictures, each 
depicting two children running, one faster than the other. 

They were then given four pictures, each one a different, previously 
shown child, and asked to order these from fastest to slowest, 

b) Coins Test: Students were shown pictures of three ficticious coins 
and their ordinal values (highest to lowest) and then were asked 
to order in value a series of pictures of various combinations of 
them. 



♦Evaluation Report l-B-6: Summary Test Data: Detroit Schools 

♦♦Evaluation Report l-B-5: Test Data on Some General Cognitive 
Skills Related to CSMP Content 
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c) Houses Test: Students were shown a series of pictures of houses 
differing from one another in number of windows (one or two), 
number of chianeys (one or two), and color (blue or red). They 
were then asked to place these pictures on a board with spaces 
for houses so that houses would differ from their neighbor along 
certain attributes. 

It should be pointed out that "relations** in this context does 
not involve arrow diagrams but rather the commonly used concepts 
of more-less, sajne-different and faster-slower. However, experience 
doing "relational thinking/' which might or might not involve y..*row 
diagrams, might be expected to improve perfomance on these tasks. 

Test 3. Analytic Reasoning: Students were shown a picture of an array of dots 
with non-overlapping loops containing between them all the dots. 
They were told a "secret dot" was in a loop which held a certain 
nximber of dots. This process was repeated, sometimes twice, with 
the same dots but new placement of loops and the student asked 
to name the unique dot which satisfied all these conditions. 

This test required students to go through a process of elimination 
in first reducing the set of possible dots to a smaller set and to 
then compare that smaller set with one or more smaller sets to 
determine which dot was common to these smaller sets. That is, 
•the student had to shift his attention back and forth between 
different subsets of dots. It may be conjectured that this ability 
to focus one's attention on certain members of a set of objects is 
enhanced by the CSMP program. 

The analysis of class means, with adjustment for pretest scores, was carried 
out in the usual way. Tlie results are given in Table 4. 

Table 4 

Analysis of Class Means 
for Three Tests of Cognitive Skills 



Test 


Number of Items 


Adjusted Mean Scores* 
CSMP Non-CSMP 


F Ratio:** 
P less than 


Class! f i cati on 


20 


li».3A 13-78 


.Ail 


Relational 
Th inki nq 


15 


10.92 10.19 


.13 


Analytic 
Reasoni ng 


9 


3.25 2.75 


. 16 



* Adjusted for pretest scores 

** 1 and 5, 1 and 9, and 1 and 7 degrees of freedom respectively 
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The differences were not significant on any of the tests, although CSMP 
classes had higher mean scores in each case, particularly the Relational Thinking 
and Analytic Reasoning tests. 

The three subtests of the Relational Thinking test were also administered 
to two CSMP and two comparison classes in Detroit* and the resulting difference 
in means scores was in favor of the CSMP classes and signj.ficant at the .05 level. 

Kindergarten Test* * 

Two 16-item tests were constructed to measure student achievement of 12 
of the 14 objectives of the CSMP kindergarten program. Eight of these 14 
objectives dealt with rather traditional concepts such as counting, numerals, 
and shapes, while the other four objectives were related to the more unique 
aspects of CSMP, such as arrow diagrams and Venn diagrams. 

The tests were administered to a total of 12 CSMP kindergarten and 8 
non-CSMP kindergarten classes in two local districts as shown below in Table 5. 
One form of the test was administered to a random half of each test. In District 
A, the non-CSMP classes were, with one exception, from the same schools as the 
CSMP classes, while in District B they were from different schools. Thus the 
comparative data shown in Table 5 may not be particularly valid. 

Table 5 

Analysis of Mean Scores 
on CSMP Kindergarten Test 





Test Form A 




Test Form B 




CSMP 


Non- 


CSMP 




CSMP 


Non-CSMP 




n 


Mean 


n 


Mean 




n 


Mean 


n 


Mean 


District A 


68 


15.1 


68 


11.6 




68 


13.8 


68 


12.6 


District B 


45 


13.7 


21 


13.2 




39 


13.9 


18 


12.7 



*Evaluation Report l-B-6: Summar>^ Test Data: Detroit Schools 

**Evaluation Report l-B-4: End^of-Year Test Data: CSMP Kindergarten 
Content 
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An item analysis of the test items was carried out according to objective 
tested and the mean percent correct across . students computed separately for 
CSMP and for non-CSMP students. The starred entries are those objectives on 
which the'mean percent correct for items related to that objective differed by 
more than 10, 



Table 6 

Item Analysis of Kindergarten Test 
According to Objective Tested 



Objectives CoHon to 


NuBber of Itcas 


Memn Percent Correct: 


Mean Percent Correct 


Most Kinderxarten Progrtas 


Testing Objective(s) 


CSMP Students 


Non-CSMP Students 


Objectives 1 and 6: Identif/ing Shapes 


3 


94 


9S 


Objectives 3 and 4; Counting Objects 


4 


95 


92 


Objective S: Identifying Numerals 


6 


96 


9S 


^Objectives 7 and 8: Story Problcas (with pictures) 


5 


90 


79 


Objective 11: Longer, Shorter 


2 


9S 


98 


*Olbj active 12: ftote Counting 


4 


eo 


€6 


Objectives More Unique to CSMP 








'Objectives 9 and 10: Arrow Diagrams (Niaeric) 


3 


40 


2S 


*Obj active 13: Arrow Diagraas (Non-Nuaeric} 


3 


61 


35 


Objective 14: Venn Diagraai 


2 


ss 


S6 



Test of CSMP First Grade Content: Mid-Year* 

This test was administered to each local CSMP first grade class at about 
the time the class reached lesson A140, whicin was sometime in February or March. 
The test was constructed from an analysis of the student workbooks and of the 
CSMP Content Resume, a sequential developme*:t of 15 content areas as they appear 
in the first grade lessons. 

Three tests were constructed to test two content areas each. Then each 
student took one of the three tests, the assignment within classes based on 
pretest score to ensure that ''equal'* thirds of each class took each of the three 
tests. The six content areas tested were the following: 

I. Number-numeral: Counting the number of a given set of objects, writing 
numerals in order, producing a given number of dots, matching numerals 
to sets, 

II. Addition: Completing addition stories (equations) given appropriate 

string pictures, completing addition tables, 
III. Relations: Drawing dots or arrows for spirals (+1, +2), drawing arrows 

to show '^greater than" relationships. 
IV. Order: Using correct symbol of <, > or = to show correct relationship 

of two numbers or pictures of things. 
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V. Minicomputer: Strictly speaking this is not a content area as such, 
but is intended as a pedagogical tool for teaching a variety of ideas. 
However, students did spend considerable time with the Minicomputer 
and it is an important part of the curriculum, however classified. 

VI. Multiplication: Various embodiments of multiplication and doubling 

appear, though sometimes very briefly, in ten of the first 136 lessons. 
Workbook practice, however, is very light. Thus it was understood from 
the outset that mastery of any of them could not be expected. Of all 
areas tested only the multiplication items were not expected to be 
well answered. 



A- summary of the data from the six content areas is presented in Table 7. 
The tests were relatively homogeneous as indicated by the high KR 20' s. The 
correlations with the CAT are in the expected range, indicating that the test 
scores are moderately related to performance on the pretest. 

Table 7 



Summary Data from Mid-Year 
Test of CSMP First Grade Content 



Content Area 


Number 
of Items 


Mean 
Percent Correct 


KR20 


Correlition 
with !^ret«st 


Relations 


24 


72 


.92 


.60 


Multiplic.-^t ion 


6 


37 


.71 


.41 


Number -Kumcra 1 


39 


90 


.92 


.52 


Addition 


IS 


79 


-93 


.50 


Min icotaput cr 


32 


8} 


.96 


.54 


Order 


IS 


90 


.94 


.4S 



Test of CSMP FirSw Grade Content; End-of-Year* 

A total of 12 content areas was tested. The items were constructed in a 
manner similar to that of the Mid-Year Test, using student workbooks and an 
analysis, in the Content Resume, up to lesson A240- Two points sKcmld.be noted. 
First, content on K«hich students had already shown a high. degree of i>roficiency 
iji the Mid-Year Test was not retested. Second, because of the spiral nature of 
the CSMP curriculum, the test items of any given content strand exhibited a wide 
range of difficulty levels, ranging from very easy items on which virtually all 
students were expected to be at mastery level, to items testing content imich 
•further along in the strand which students had ver>' little experience with 
and which only the better ones might get- 



♦Evaluation Report l-B-2: End>>of-Year Test Data: CSMP First Gr ade 
Content " ' 
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The tMt was administered tp all local CSMP first grade classes in May. 
Within each class, half of the students were randomly assigned to take six 
of the subtests and the other ha;f of the students took the other six subtests. 

Table 8, below, summarizes the results of the testing. For each content 
area, a sample item has been provided and the ratings given by the evaluation 

- — This has been done 



staff to the performance of students on that content area. This has been done 
on a quartile by quartile basis according to the scores on that pa rticular subtest, 
that is, the performance of that quarter of the students who scored highest on the 
subtest has been rated separately, as has the second highest scoring quarter of 
students, etc. The ratings were subjective in nature and took into account, besides 
the percentages correct for students, the number of lessons and amount of work- 
book practice devoted to that content, the level of difficulty of the test items, 

^ — '"^ For 



the concept of "carrying," than on items testing skill in one-digit multiplicat 
which is a highly spiralled topic presented in many embodiments and with little 
emphasis on skill building in the first grade 



ion 



Table 8 

Siimmary Data from End-of-Year 
Test of CSMP First Grade Content 



Subtest 
Subtest 


Saaple Itcn* 


N(»ber 
of I teas 


Mean ^ 
Correct 


Rating of Perfo 
According to Subt 


mance* 

est Sec 


• 

res 










Top Quarter 
of Students 


u 
t> 

*^ </) 

u *>• 

rt r. 
r> o 
cy "J 

C -7) 
O 

o 

C' o 

in 


Third Quarter 
of Students 


Eotton Quarter 
of SfjJer.ts 


a 

z 

c 


Multiplication 


3 X 4 * 


10 


54 


VG 


VG 


I 


I 


A 


Integers 


2 ♦ 3 « _ 


8 


66 


VG 


VG 


A 


I 


A 


lUtionals (a) 


Shade H of each 
CAVe cut in half 


11 


82 


VG 


VG 


VC 


A 


VC 


lUtionals Ih) 


»5 X 4 « 


S 


49 


VG 


VG 


VP 


VP 


A 


Counting Honey 




s 


62 


VG 


VG 


A 


VP 


A 


Mlnicppptitrr 


lUr M.C. for 


17 


SI 


VC 


A 


A 


I 


A 


Relations* ^ 

Addition 
Subtraction 


10-1 - _ 


27 
IS 
13 


60 
77 
81 


VG 
VG 
VG 


A 

VG 

VG 


A 

VG 
VG 


I 
A 
A 


A 
A 
VG 


Order ^J^,- 




6 


64 










A 


Probability 


'♦air* Si'titftcr 


3 


S4 










A 








Not Rated 




Venn Diagrams 




1 


39 




1 








•It«is shown 


in Buch abbreviatea 


fom. 















••VG-V«ry Good, A-Ad»qu«te, l-lnidequate, VP-V«ry Poor 
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Although there is some concern about those subtests which received poor 
ratings for the lower scoring students, and although there were large differences 
in performance between highest and lowest scoring students on some subtests (not 
shown in Table 8), it is concluded, for the present, that students' overall 
performance on this test was adequate. This statement is made with reservations, 
at least until one can determine whether or not the low scoring students, as 
they progress through the curriculum, show improved performance on these tasks. 

For each subtest the class means for the 14 local first grade classes who took 
the test were adjusted for entering ability. Rather consistent int«rclMs 
differences were then found across many of the subtests and these differences 
could not be accounted for in any significant way by any of the following 6 
variables (derived froia observational or questionnaire da^a) : 

a) degree of teacher adherence to the program 

b) teacher skill in CSMP recoomended techniques 

c) student involvement or attentiveness 

d) number of lessons completed 

e) teacher attitude towards program 

f) student attitude towards "math*' 
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Teacher Reactions to CSMP 
Teacher Questionnaires* f 

Two questionnaires were sent to all pilot teachers. The Midyear Questionnaire 
was sent in January and 79% of the teachers responded. The End-of-Year Question- 
naire was sent in June; the response rate was 56-5. The lowei figure for June may 
be due to the lateness with which the questionnaires were sent out. Since the 
later questionnaire was more extensive than at mid-year as well as repeating 
virtually all the earlier items and since later responses on identical items 
tended to be similar or only slightly more positive than jtrlier responses, 
only the data from the End-of-Year Questionnaire is siimmarJzed here. It was 
also true that based on an analysis of mid-year returns, the later respondent 
group was very representative of the larger group k'ho responded at mid-year. 

Since the questionnaire was quite lengthy, and requested several free- 
response answers, it is difficult to adequately summarize the data in a few 
paragraphs. Thus only what are considered to be the major findings are given 
in point form below. For the most part, all groups of teachers, local and 
outer ring, kindergarten and first grade, responded very favorably to the 
program and liked to teach it. Very few questions drew iftany negative responses. 

a) CSMP was overwhelmingly preferred to previously used mathematics 
programs on student interest, achievement and overall quality. 
Out of a total of 181 responses to these three questions only 2 
were negative and 11 were neutral. 

b) Of 39 responding teachers who were planning to continue teaching 
next year, 36 said they would continue with CSMP. 

c) Teachers were asked to write a free-response evaluation of the program. 
Seventy-seven percent were judged favorable, 9% unfavorable. 

d) Forty-eight of the fifty-five responding teachers listed more areas'" of 
increased student accompiishnent tlian areas of decreased accomplishment. 

e) Outer ring teachers, trained by thei^ own coordinators, responded less 
positively to certain items than did local CSMP-trained teachers. They 
found more difficulty in teaching certain content, progressed less far 
in the sequence and rated lower the "appropriateness of CSMP for low 
ability students." 

f) Teachers tended to estimate the expected "life" of various instructional 
materials to be shorter than recommended by CSMP, particularly demon- 
stration materials. 

g) Teachers were asked a series of questions about each of five ^najor 
content areas. For these five areas, teachers felt that they had "few" 
or "no problems" in presenting the material, that their students "liked 
the topic moderately" or were "enthusiastic about it," but in about 30% 
of the cases, stated that there were ."difficulties which required extra 
tiae to resolve. 

h) AbMEt one-third of the taachers felt that "ease of managing the various 
Materials in a CSMP classroom" was "poor" and many of these teachcTs agreed 
with the statement that "student workbooks and worksheets should be bundled 
as a set for each student." Kindergarten teacher, howerver, felt that the 
materials would be managed adequately. 



♦Evaluation Reports 1-C-l: Teacher Training Report and 
l-C-3: Mid-Year Data from Teacher Questionnaires 
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i) The first grade lesson sequence contains 308 lessons and it is 
recommended by CSMP that teachers complete at least the first 
240. Many teachers did not reach lesson 240 and the median was 
only 248 for local teachers and 210 for outer ring teachers. 
Roughly half of the teachers thought the sequence was ''too rapid/* 
one-third taught math twice a day and the median number of minutes 
of daily math instruction was 45, Fifty-four teachers thought that 
the time required for daily preparation, compared to their previous 
math program, was either ''less" or '^about the saiTie'' or "more now but 
would be about the same after a year's experience." Only seven 
teachers thought the time was "more and would continue to be." 

j) Twenty-six percent of the respondents thought CSMP was less or much 
less appropriate for low ability students than was their pre*^iously 
used math text, while 36% thought it was about the same and 38% thought 
it more or much more appropriate. 

Classroom Observations* 

Each first grade classroom was observed at least four times in order to: 

a) quantify the faithfulness of the program's implementation 
in the various classrooms vis-a-vis the recommendations' of 
CSMP . 

b) provide a qualitative description of the program in operation 
with a variety of teacher styles and classroom settings. 

Implementation Form 

An implementation form was developed which contained a nine point scale 
yielding separate scores on three variables: the degree to which the presented 
lesson adhered to the lesion plan, teacher skill in using certain teaching techniques 
and student reactions. Also recorded at the time of each visit were the teacher^s 
comments during a brief interview at the end of the lesson, the observer's notes 
and the time taken to feach the lesson, ' 

In the 59 classroom observations for which the implementation form was used, 
adherence to program was very high. For example, in 95% of the observations 
materials were used as intended and no content er-rors were observed, and in 80% 
of the observations the lesson plan was followed with reasonable care. 

In the area of teacher skill in using certain teaching techniques, three of 
the items dealt with "mechanics and preparation," "focus of the lesson" and 
"questionning technique," For these three, 66% of the ratings were "good" and 
8% were "poor," However, on a fourth item, "pace of lesson," about one-third of 
the lessons were rated as '*too slow." This item also showed the most variation 
among teachers. 



♦Evaluation Report l-C-2: Observations of CSMP First Grade Classes 
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In the area of student reactions, where were two items, dealing with 
student attentiveness and student involvement. For both of these items 12% of 
the ratings were '4ow** and 30% were ^'high/' 

Thus in general the prograro was being implemented as intended. For only 
one teacher could the faithfulness of implementation be considered even question- 
able. The only significant problem seemed to be that the pace of the lescon 
often seemed rather rlow. Independently, it was noted that the required time 
exceeded the recommendeJ tijne by an average of 8 minutes [where lessons were 
generally from 12 to 20 minutes in recommended length). 

Summary of Observer Notes and Interviews 

a) In general, most of the teachers observed implemented the program 

in an acceptable manner. Although problems were encountered by some, 
most teachers were able to cope. 

b) The program appeared to vary as the makeup of the classes and teacher 
styles varied. Thus, the program does not seem to be either "teacher 
free'* or "student free." 

c) The program seemed quite adaptable to differing classroom situations. 
That is, although the program varied from class to class, it appeared 
workable in most cases. 

c) Teacher attitudes were generally quite positive toward the program 
throughout the year. Generally, the positive reaction appeared to be 
based on an enthusiasm for the mathematical ideas in the program and 
on the reactions of the children. 

d) The students in the classes appeared able to perform in a reasonable 
fashion during the lessons. The children in most classes seemed at 
least moderately interested and involved in the lessons. 

e) Many teachers had trouble sticking to the recommended pace and this 
appeared to be more dependent on teacher style than on class ability 
level. Teachers who moved along briskly seemed to encoionter fewer 
problems and to be happier about the course. 

f) Many teachers expressed concern thvt the program was too hard for low 
ability students, though most agreed that their slow students were 
learning as much as ever« In tht observers' opinion, this reaction 
may be due in part to the spiral nature of the curriculum and the real 
problem may lie in teachers* ability to deal with a wide range of 
abilities when using a spiral curriculum. 

Interviews with Kindergarten Teachers* 

Near the end of the school year a structured interview of 20-30 minutes 
duration was conducted with each of the 15 local kindergarten teachers. Tlie 
interview was conducted at the end of a mathematics lesson, which was observed 
by the interviewer. Bearing in mind that these observations were not very 
extensive, the observer- interviewer noted: 

"First, in all classes there was evidence that, as a group, the children 
seemed able to do the lessons. Most of them appeared neither lost nor bored... . 
Second, all of the teachers appeared to be implementing the program in a 
reasonably faithful manner. .. (they) seemed to be generally able to get the 
concepts across to the children and to actively involve the children in the 
lessons." 



EKLC 



*Evaluation Report l-C-5: Interviews with CSMP Kindergarten Teachers 

2o 



19 



Each teacher was asked a total of 25 questions dealing with 10 aspects 
of the program. These aspects and the responses to them are summarized very 
briefly below. 

1. General Impressions: Most teachers said their overall attitude toward 

the program had become more favorable since beginning' the program. Twelve 
of the 15 teachers were very positive towards the program, the other three 
were either neutral or only slightly positive. 

2. Program Comparisons: A total of 29 comparative statements were made and 
25 of these were favorable to CSMP vis-a-vis other kindergarten programs. 
The features of CSMP most commonly cited as distinctive were the higher 
level of student enthusiasm and the challenging nature of the content. 

3. Best and Worst Aspects of CSMP: Forty-four responses were given for 
"best asperA;s/' with 21 different aspects noted. The most common 
responses were related to games, rods and A-blocks. Nineteen responses 
were given for "worst aspects" the most common responses being that CSMP 
was "hard for slow children" and that arrow diagrams ^ere too difficult 
for children to draw. 

4. Child Attitudes: Four times as many responses were given for "things 
children liked" as for things children disliked" about CSMP. Most 
teachers indicated children's attitudes were more favorable towards 
math. 

5. Student Learning: All teachers considered that overall learning was 
adequate, though about half indicated that some "slow" children had 
difficulties. Two-thirds of the teachers thought children learned 
more this year, 

6. Parent Reactions: For all teachers, parent reaction was limited and 
generally quite favorable towards the program. 

7. Quality of Teacher Lesson Guide: There was wide disagreement on this 
issue, particularly the rather unstructured nature of lesson sequence, 
which some teachers saw as a strength and others as a weakness. 

8. Changes and Supplements to the Program: About half the teachers provided 
supplementary materials or lessons on elementary counting and number 
concepts. Other changes or supplements to the program were minor and 
isolated. 

9. Management of Materials: Thirteen of the fifteen teachers indicated there 
was little or no problem in managing materials. 

10. Teacher Preparation: All teachers agreed that some sort of workshop should be 
provided for beginning CSMP teachers. Several stated that a better overall 
picture of the program and where it was going should be provided, and that 
teachers should be instilled with more confidence in their ow ability to 
teach the program. 
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Teacher Logs* 

All pilot teachers were asked to keep a daily record of the lessons taught 
and the time required and their rating for each lesson. They were also asked 
to make weekly free-form responses they thought relevant regarding some general 
aspects of CSMP. 

Lesson Ratings: Each lesson was to be rated **Good,'* "F:iir/' or *Toor.** Table 9 
shows the number of ratings put into each category when the ratings were summed 
across teachers and lessons. 

Table 9 



Number of »^Good," "Fair»» 
or *'Poor" Ratings 





Total Number 
of Ratings 


Number of Times 
Lesson Rated "Good" 


Number of Times 
Lesson Rated "Fair*' 


Number of Times 
Lesson Rated "Poor" 


First Qndm 
Teachns 


7,244 


6,125 


1,037 


82 


Kindergmrten 


2,573 


2,109 


407 


57 



At both the first grade and kindergarten levels, over 85% of the lessons 
received "Good" ratings from most (>70%) of the teachers and only one lesson 
at each level did not receive a majority of "Good" ratings. The first grade 
content areas generally having the highest rated lessons were Integers, 
Geometry, Probability, Workbook Lessons and Eltimation, while lessons having 
to do with Combinatorics and, for local teachers. Subtraction and Mental 
Arithmetic, were lowest rated. 

Teachers were asked to tell what they did for students who entered their 
class during the course of the year. Listed about equally ofcen were: individual 
help by teacher, help from other students, and catching up with earlier work- 
sheets and workbooks. Teachers rarely responded to the question regarding 
specific problems encountered. 

Responses to questions regarding changes of sequence, the use of 
Free Days, the use of supplementary materials, suggestions for changes and 
anecdotes were very sparse and are not sunimarized here. 



*Evaluation Report l-C-6: Analysis of Teacher Logs 
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Other Issues 

Cost 

Estimating the cost, on a per pupil per year basis, of adopting a curriculur. 
is a notoriously difficult enterprise. There are several reasons for this. Items 
advertised as '^additional materials" may be sometimes just that, and not at all 
necessary to a classroom teacher; at other times the/ are really integral parts 
of the curriculum, such as testing materials in an individualized program. Some 
materials are consumable, others are not but their length of service is not 
consistent from district to district, depending on the kind of usage they get. 
Occasionally duplicating masters are offered in which case a school must bear 
labor and paper costs. Teacher training may or may not be necessary depending on 
teadier experience. Teacher aids may be advisable on a ratio of between 1 to 20 
or^ to 200 students. There may be savings in cost for large numbers of classes 
at any grade ?..evel, with demonstration materials for example, or by the use 
of a specialist teacher. Some curricula require considerable expenditure of time 
on the part of a district "coordinator" or "curriculum consultant." 

The complexity of any of the above items varies from curriculum to curriculum 
and from district to district. In attempting to calculate the costs associated 
with CSMP a further confusing issue is the fact that present costs of producing 
materials are based on relatively small scale production methods and with 
rather unsophisticated methods of packaging. Conversely, the usual costs of 
advertising and selling are not now built into the price of materials. In fact 
the amount of money paid by pilot sites for materials is virtually the exact 
amount presently required to produce and ship these materials. 

There are three basic kinds of materials required for CSMP: student workbooks 
and worksheets, teachers' guides (primarily sets of lesson plans) and other 
materials, either classroom demonstration materials or student manipulatives . 
Of these three, the student workbooks and worksheets are by far the most 
expensive and are also the only consumable items. Table 10 shows present cost 
figures, for starting up the program and for maintaining it after the first year, 
for kindergarten through second grades. Maintenance costs are calculated by 
dividing the cost of each article by the expected number of years of life. Costs 
are presented on a per pupil basis and a class size of 30 students is assumed. 
Optional materials are not included; they would add only about $.25 per year per 
student. The figures were computed from information in the CSMP Order Form (1974-75) 
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Table 10 

Per Pupil Per Year Costs 
of X-2 CSMP Materials 
For Current Production Methods 





Kindergarten 


First 


Grade 


Second Grade 




Start Up Maintenance 


Start Up Maintenance 


Start Up 


Maintenance 


Student Materials 

(Worksheets and Workbooks) 


$.92 


$.92 


$3.93 


$2.93 


$3.72 


$3.72 


Other Matfr:*als 

(Teacher Guides and Manxpulatxves) 


$.32 


$.08 


$1.95 


$ .85 


$1.58 


$ .46 


Total 


$1.24 


$1.G0 


$5.88 


$4.78 


$5.20 


$4.18 



In order to get an estimate of costs for other elementary level mathematics 
curricula, the most recent catalogs of most 'of the leading publishers were 
analyzed. The estimates given in Table 11, page 23, are based on that information. 
Where optional materials are available, they are not included in the cost unless 
they are clearly an integral part of the program. In fact there is a 
bewildering array of multi-media supplements available, some quite expensive 
which could easily form the basis of a math program. Duplicating masters are 
occasionally available in place of or as well as consumable naterials. Student 
texts are sometimes in hard cover, accompanied by a work tablet, and sojnetiuies 
sold as a consimble text-woTkbook, Sometiikes it is not clear what materials 
are necessary for starting up a program and what are necessary for maintenance 
of it. In mking the calculations for Table 11, a class size of 30 has been assmed 
and maintenance costs have assumed a li'^e span of four years for all non- consumable 
materials. This may be a generous estimate for some materials and stingy for others. 

Thus, the entries in Table 11 are only estimates and many factors have been 
taken into account. Except for some of the more expensive programs, the estimates 
do not include the cost of any manipulatives or supplementary materials. 

Bearing in mind the fact that the CSMP costs are based on present production 
methods for pilot study use, while costs for other programs are based on catalog 
prices, some comparisons can be made. It can be seen that at the kindergarten 
level CSMP is cheaper than most other programs. At the first grade level about 
two-thirds of the programs listed, those consisting of a basic consumable work- 
book possibly augmented by practice exercises in one form or another, cost between 
$2 and $4 per pupil per year for both start up and maintenance. Thus CSMP is 
more expensive than most and cheaper than some. About the same thing is true at 
the second grade, though CSMP costs are down some from first grade while some 
other programs are slightly higher than in first grade. 
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Table 11 

Estimates of Per Pupil Per Year 
Cost for Various Mathematics Prograjns 
at First Grade Level 





First 


Grade 


Kindergarten 




Start Up 


Maintenance 


S U 


M 


Publisher and Text 
• 


Costs 


Costs 


C 


C 


Encyclopaedia Brittanica 










Math Workshop 


$2.20 


$1.70 


$1 .25 


$1.25 


Silver Burdett 










S.B. Mathematics System 


8.70 


5.10 


5.00 


3.00 


Modern Math Through Discovery 


2. 60 


2.50 


2.50 


1.85 


Science Research Associates 










SRA Mathematics Learning System 


3.60 


3.40 


1.50 


1.50 


Ginn and Company 










An Applied Approach 


3 . 60 


3 . 30 


3,45 


2.00 


Individualized Mathematics System 


4.60 


2, 10 






Essential Modem Mathematics 


5.10 


3.15 


2.50 


.60 


Scott Foresman 










Mathematics Around Us 


3.15 


3. 10 


1 . 7U 


1 . /o 


Basic Mathematics Program 


2.50 


2.25 


2.50 


2 .50 


Rand McNally 
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iNcvv VrfCiiLUxy cau^d LX und X ^uxpuxduxuii 










TPT Ma1"h pm a1" 1 p<> 


7.85* 


7.85 






Laidlaw Brothers 










Understanding Mathematics Program 


2.20 


2.05 


1.50 


1.50 


New Laidlaw Mathematics Program 


2.45 


2.20 


1.50 


1.50 


Cuisenaire Company of America 










Opening Doors in Mathematics 


7.80 


5.70 






American Book Company 










Mathematics Target System 


2.90 


2.70 


1.75 


1.75 


Mathematics in Action 


2.55 


2.50 






Addison-Wesley 










Investigating School Mathematics 


4.12 


4.00 


1.65 


1.55 


Elementary School Mathematics 


2.70 


2.60 


1.65 


1.55 


Harcourt Brace Jovanovich 










Harbrace Mathematics 










Elementary Mathematics Series 


3.55 


2.60 


2.72 


2.10 


2.70 


1.90 


2.00 


1.50 



*Exclusive of associated personnel costs. 
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Another cost which an adopting school district must carry is that of 
training teachers. No dollar figures are given here because that cost is 
too dependent on the role of the coordinator, the size of the adoption, the 
extent of teacher training desired, and many other factors. It would seem 
reasonable however, that these costs would be moderate since it is a one-time 
(per teacher) training workshop, which might be carried out by an already 
in-place district-wide coordinator and might require only 2 or 3 days pre- 
service and perhaps an hour per month during the first year. The important 
consideration is probably the availability of a coordinator whose responsibility 
is district wide; if such a person is available to a district, training costs 
should not be high. 

External Review* 

The Mathematical Association of America, in response to a request on behalf 
of CSMP by Dr. Robert Dilworth, Chairman, CSMP Evaluation Panel, supplied a list 
of 'iiaices of persons deemed suitable to evaluate the work of CSMP. Any person 
on the list who had been previously involved with the program, or who was known 
to have expressed any kind of opinion about the program, or who was known to be 
unavailable as a reviewer was excluded from consideration. Five persons were 
selected and asked to review the materials. One was unable to accept and his 
place taken by another person recommended by the MAA. The five reviewers were 
the following: 

Professor Shirley Hill 

University of Missouri at Kansas City 

Professor Dan E. Christie 
Bowdoin College 

Professor Leonard Gillman 
University of Texas at Austin 

Professor George Spiinger 
Indiana University 

Professor Sherman Stein 
University of California at Davis 

The reviewers were sent all K-2 materials except manipulatives (some 
second grade workbooks were not available), sample third grade materials, 
sample fourth grade lesson plans, and some background explanatory materials. 
Each was asked to give his '^opinions as to the soundness and relevance of 
the mathematical content .. .The format for your review is open for you to 
respond as you see fit/* 
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Dr. Shirley liill agreed to serve as chairman of the group and to summarize 
the set of reviews. Her su]ninar>' review follows, in toto . 

"If this sample of mathematician's opinion is in any way representative 
then I cannot help but comment that the mathematical commxanity is a long way 
from any consensus concerning what mathematics is important and what should 
be taught. (The possibility that there could be agreement on how it is taught 
is so remote as to exclude hope.) The difficulty of summarizing the five reports 
is exacerbated by the apparent fact that the reviewers' perceptions of their roles 
and the purpose of their evaluations differed greatly. The reports seem to be 
addressed to different audiences and vary widely in degree of specificity, in 
focus and in the framework of time and vision (farsighted, shortsighted, near- 
sighted, hindsight, foresight, the ''now," the future, the past, etc.) within 
which any value judgment is imbedded. Thus I strongly urge any reader of this 
summary at least to skim each of the individual reports. 

"The overall impression of the materials was favorable; three reviewers 
expressed quite favorable evaluations directly, the reaction of another was 
mixed, and the impression of the fifth cannot be said to be favorable, though 
it was not explicitly negative. 

"One point of general agreement in the reports was on the soundness of the 
mathematical content. The material is seen to be mathematically soxand without . 
any egregious technical or conceptual errors. There were differences of opinion 
concerning matters of preference and taste in the development of the mathematical 
ideas . 

"It was at least implicit in every report that it was impossible' to separate 
completely in an evaluation of this kind, matters of mathematics and matters of 
pedagogy. Certainly most of the differences in preference concerning the way 
the mathematics was presented had little to do wrth mathematical soxandness but 
rather related to questions of learning, development, concept formation and the 
like. Many of these are empirical questions. I think that it is fair to say 
that most of the very specific comments and specific criticisms concern 
psychological and pedagogical issues. 

"An example of a curricular element which is a mix of mathematical and 
pedagogical issues is the use of the minicomputer. This is the single point of 
complete agreement among all reports. There is too much reliance on the 
minicomputer. Three reviewers vehemently opposed its use as an aid altogether; 
the other two seriously question its value in light of the very great investment 
of time. (Both of these reviewers agree that the effectiveness of the device 
with respect ot computational skills is an empirical question) All five 
reviewers are dubious to very negative on the minicomputer's mixture of a 
binary and decimal base. 

"Are the materials innovative, current, timely? Comments ranged from 
'it is more of the same' to 'the material is refreshingly full of new ide.as.' 
The majority were of the opinion that the materials were timely and current 
and in many instances excitingly new. One reviewer foxond much new material 
of which he could approve but too much 'old' material from the era of 'new math.' 
One fotind some 'good sections' but little mathematics and much 'obsessive ritual.' 
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"The question of relevance is tricky, as everyone knows. 'Relevance' 
has no meaning except in the context of one's objectives, values, indeed 
one's philosophy. I can only infer that there are differences among the 
reviewers in the philosophical basis of their views of mathematics ~ what 
it is and what it does. Thus it is impossible to summarize the comments 
relating to perceived relevance of the material. There simply is no constant 
base for the opinions expressed. Certainly I can ascertain no consistent set 
of criteria for relevance. 

'*Let me offer some examples of these differences. One reviewer sees 
the authors of the materials as 'oriented to pure mathematics' and working 
in the 'format of the past twenty years,' while another feels that the extent 
of 'student's participation' and spontaneity is encouraging, apparently viewing 
the materials as having moved beyond 'the precocious discussions o. systems 
and structure' of the past decade. 

'*One reviewer sees too much carryover of material from the 'new math' 
(I defy anyone to provide a clear-cut definition of that unfortunate term) 
and views such material as faddish while another, believing in the need for 
more historical perspective in distinguishing trends from fundamentals, 
compliments the authors on maintaining a balanced program that is timely and 
relevant today without discarding all the achievements of recent years. 

"The majority of reviewers saw the materials as modem, relevant on 
today's trends in mathematics and its applications with potential for 
developing competent future mathematical users and problem-solvers. 

'*I will end by mentioning some specific things mentioned in more than 
one review. All reviewers praised the inclusion of extensive study of 
probability. Most liked the material on relations and functions, on grapliing 
and arrow diagrams, on combinatorics. 

"Three reviewers specifically pointed to the 'spiral' development and 
saw this as a positive feature. These three reviewers also believed the 
balance between concepts and applications was good. Two specifically pointed 
out that the activities stimulate active problem-solving and logical reasoning. 

"Most reviewers were critical of the material on sets, set operations, 
and Venn diagrams. Two opposed the material on the properties of arithmetic 
operations. Two felt there should be more reliance on manipulative, physical 
materials . 

"As mentioned earlier,. all reviewers were negative (in varying degrees) 
about the minicomputer," 

Continued Usage, Second Year of Pilot Trial 

One very good indication of satisfaction with CSMP in a pilot site is the 
decision that the district makes regarding continued usage of the program after 
a year's trial. Table 12 reflects these decisions. 
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Table 12 

Number of Pilot Classes, By District 
1973-74 and 1974-75 
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*T: Teacher, P: Principal, U: University Based Person, 
C: District Wide Coordinator or Consultant 

**This coordinator worked with two different school districts. 
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When looking across districts at the increase in number of pilot, classes 
from 19.73 to 1974, it is apparent that there was a high degree of satisfaction 
with the program. In September 1973, 138 classes began using CSMP materials. 
The next year, this number had more than tripled to 448, exclusive of classes 
begun at the cwo nev sites. 

; It is interesting to look at these decisions as a function of the coordinator's 
usual role within the school system: 

a) Teachers, Five CSMP pilot teachers also took on the role of the 
coordinator. In two of these districts the program was discontinued, 
and in none of the other three was there a large increase in the number 
of classes (nine or more additional classes) or even a new start at a 
second school, 

b) Principals. In four of the five cases in which the coordinator was 

a principal, the numbers of classes using CSMP in 1974 were about the 
same or slightly higher than the number in 1973, In none of these 
four cases was a start made at a new school. In the fifth case, 
there was an expansion from nine classes in one school to 18 classes 
in seven schools. However, all 18 classes are kindergarten classes, 
the three first grade teachers from last year having discontinued the 
program with no new first grade classes added. 

c) College Based Personnel. In four of the five cases where the coordinator 
was based at a local college, the program was tried in only one school 
(three of these were lab schools). For these four sites, either 

the program was discontinued or there was very moderate expansion. 
In the fifth case where the program was begun in two schools, there 
was a large expansion from six to 36 classes. 

d) District Wide Coordinators. Where the coordinator had district-wide 
responsibilities as Curriculum Consultant or Curriculum Coordinator 
or some such title, there was generally a much more dramatic increase 
in the number of pilot classes. In all seven cases where the program 
was begun in more than one school, and in two of the six cases where 
it was begun in exactly one school, there was a large expansion of 
CSMP in the district. 

Thus it appears that the coordinators who achieve the most impact for 
CSMP are those with full-time district-wide responsibilities. It is also true 
that when the program is started in a single school it does not usually get 
used beyond that school; where the program is used originally in more than one 
school, there is likely to be a large expansion the next year. One suspects 
this situation is similar with other new progi'-ams and in other subject areas. 
Table 13 illustrates these trends for CSMP. 
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Table 13 

Instances of Large Expansion 
from 1973-74 to 1974-75 
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Discussion 

Before summarizing the major findings of the first year of the Extended 
Pilot Trial it is worth recalling some of the provisos noted earlier. In the 
first place, pilot school districts selected the program on a voluntary basis 
and made their own decision as to the number of pilot classes and which teachers 
would teach those classes. The teachers volunteered in some cases and were 
"designated" in other cases. Although there is some evidence, at least lor 
classes in the local area, that pilot teachers were not too unlike other 
teachers in their respective districts, the fact remains that they were a 
selected or self-selected group of teachers. They may also have been rather 
enthusiastic about teaching a new curriculum with lots of new things in it. 
On the other hand, these teachers were required to learn much new material: 
A whole new set of lesson plans; considerable use of arrow diagrams, the 
Papy Minicomputer, and Venn diagrams; and the spiral approach. First grade 
teachers were teaching students who did not have CSMP in kindergarten. Materials 
were occasionally late in arriving or arrived in disorderly condition. These 
factors must have caused some difficulties and at least dampened somewhat any 
potential invalidity due to the Hawthorne effect or the selection factor. 

In the author's opinion, the above considerations were not of major 
importance and the conclusions drawn below are accurate descriptions of 
the program in normal use. During the second year of the pilot study a wider 
range of teachers will be using the materials and most of the 1973-74 pilot 
teachers will be using the materials the second time through, . Thus many of 
the key issues will indeed be investigated under more normal conditions. It 
Kay very well be true that the existence of an industrious coordinator, with 
district^wide responsibilities and with experience in helping and training 
tMchers» if such nore important than all of the above factors, 

Th© sajor impressions that the author of this repo*c has gained from this 
first year of the Extended Pilot Trial are the following: 

1. The prograjn was very well liked. Across districts which used 
CSMP in 1973-74, there has been a three fold increase in the 
number of pilot classes for 1974-75 (p,26) and .these. districts 
have had to bear the responsibility for paying for t\.e materials 
and training teachers. Teacher attitudes towards the program 
were favorable and they overwhelmingly preferred it to their 
previous mathematics program along dimensions of student interest, 
achievement and overall quality (p. 16 and 18). This endorsement was 
much less overwhelming in the particular area of appropriateness of 
CSMP for low ability students where 38% of the teachers thought it 
more appropriate than their previous mathematics program, while 26% 
thought it less appropriate (p. 17). 

2, The program is reasonably practical. The present per pupil cost of 
CSMP is higher than the cost of the consumable workbooks offered in 
many math series and is lower than some of the more recent series 
with "individualized** materials or with extensive manipulatives (p. 22). 
Teacher training time varied from a day to a week sometime before the 
beginning of school, and this was usually augmented by periodic 
sessions during the year (p. 2). The only teachers who were observed 
carefully were those trained in a one week session and they appeared 
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to be implementing the program successfully in a variety of settings 
and in the iutcnued manner (p. 16). Materials were generally manageable, 
though some teachers expressed difficulty in keeping track of everything 
(p. 16). Most teachers took about the usual amount of time (45 minutes 
per day) but many were not able to complete even the minimum prescribed 
lessons during the cour*;e of the year (p. 17). 

3. The intrinsic merit of the program was viewed with mixed reactions (p. 25). 
A five member panel of experts independently reviewed the available 
materials and gave widely divergent opinions about the program, A box 
score of these reviews might be: favorable overall with roughly three 
for, one against, and one mixed. All reviewers agreed that the 
mathematical content is sound, but there was also agreement that there 

is too much reliance on the minicomputer. 

4. Student learning was excellent in some cases, adequate in others. CSMP 
classes generally outperformed comparison classes on a series of test 
scales designed, without regard to the content of CSMP, to measure 
student achievement cf the skills and concepts traditionally covered 

in first grade mathematics (p. 7). CSMP classes had higher scores than 
non-CSMP classes, though not significantly so, on three "content-free" 
tests (p. 9). These tests consisted of tasks which were iinfamiliar to 
all the first grade students tested and which were thought to be related 
to certain CSMP "ways of thinking." On a test of the specific content 
of the first grade CSMP program, administered only to CSMP students, 
student achievement was judged to be adequate overall although a 
signii-cant proportion of the students scored poorly on specific subtests 
(p. 13). Judgments regarding the degree of "success" attained on this 
latter test were difficult to make because of the lack of available 
standards that could be applied. 



